The invention relates to a method and a corresponding apparatus that allow a computer-controlled and/or autonomous vehicle to communicate with its environment, particularly with human beings in its environment.
At present, various initiatives on the subject of “Intelligent Transportation Systems” (ITS) at the European standardization authorities ETSI and CEN, and also at ISO in the USA, among others, involve the standardization of SAE and IEEE communication protocols and cooperative security applications that are intended to allow cooperative and, where possible, accident-free driving across vehicle manufacturers in the future. Cooperative security applications also include collision avoidance applications and collision mitigation applications, among others, for side and rear end collisions. The security applications addressed, and also the associated transmission protocols and data formats, are documented in ETSI standard TS 102 637 and in SAE standard SAEJ2735. Thus, the TS 102 637-2 standard defined what is known as a Cooperative Awareness Message (CAM) that is sent at periodic intervals by an ITS station (e.g. by a vehicle) in order to inform other ITS stations (e.g. other vehicles) in the surroundings about selected information (e.g. speed, acceleration and/or position) from the sending ITS station. The information interchanged between the ITS stations, e.g. on the basis of CAM messages, can be used in the respective ITS stations to identify collision risks and, if need be, to initiate suitable countermeasures (e.g. warnings).
The communications methods within the context of ITS are aimed at communication between different ITS stations (i.e. different machines and different electronic equipment). Direct communication between machine and human being is not taken into consideration in this case. In particular, no communication between computer-controlled/autonomous vehicles and human beings in the surroundings of the vehicle is taken into consideration and rendered possible. Such communication can be used to increase the safety and acceptance of computer-controlled/autonomous vehicles in road traffic, however.
It is an object of the present invention to allow direct communication between a vehicle (e.g. an automobile, a truck or a motorcycle) and its surroundings, particularly with a human being in its surroundings.
This and other objects are achieved in accordance with embodiments of the invention.
According to one aspect, a control unit for a vehicle is described. The vehicle can be a single-track or dual-track road vehicle, particularly an automobile or a truck. The vehicle may be set up to move in road traffic without the intervention of a driver. In other words, it may be a computer-controlled vehicle and/or an autonomous vehicle.
The control unit is set up to receive environment data from one or ore ambient sensors of the vehicle. The environment data can include information concerning surroundings or an environment of the vehicle. The one or more ambient sensors can be a camera, a laser, an ultrasonic sensor and/or a radar sensor. Furthermore, the control unit may be set up to receive position data from a positioning unit (e.g. from a navigation system) of the vehicle. The position data can be used to position the vehicle relative to a road arrangement and/or relative to other road users.
The control unit is additionally set up to take the environment data (and if need be to take the position data) as a basis for detecting at least one road user in an environment of the vehicle. In particular, a human road user can be detected. The control unit may also be set up to detect a multiplicity of different road users at different positions in the environment of the vehicle at the same time.
Furthermore, the control unit is set up to take the environment data as a basis for determining whether there is a need for communication between road user and vehicle. There may be a need for communication particularly when, in a present traffic situation, communication can avoid or resolve a potential future or already existent hazard situation, a potential future or already existent misunderstanding and/or a potential future or already existent deadlock. Similarly, there may be a need for communication in situations in which communication generates trust or acceptance in the (highly) automated vehicle and/or promotes safety or fluid traffic.
In particular, the environment data can be taken as a basis for determining that the road user is in the process of ascertaining whether be has been detected by the vehicle. In other words, it is possible to identify that the detected road user seeks contact with the vehicle (e.g. with a driver of the vehicle) in order to ensure that be has been noticed by the vehicle (or by the driver of the vehicle). Such a situation exists e.g. in the case of a pedestrian wishing to use a zebra crossing who, before crossing the road, glances at the approaching vehicle in order to ensure that the vehicle has noticed (i.e. detected) him.
In order to determine whether there is a need for communication between road user and vehicle, the control unit may be set up to take the surroundings data (e.g. to take image data) as a basis for detecting eyes of a human road user. Furthermore, the control unit may be set up to take the detected eyes as a basis for determining that there is a need for communication between road user and vehicle. In particular, the detected eyes of the road user can be taken as basis for determining that the road user is in the process of ascertaining whether be has been detected by the vehicle. By way of example, a glance from the road user in the direction of a windshield of the vehicle may be an indicator that the road user is in the process of checking whether be has been detected by the vehicle.
The control unit is additionally set up to prompt one or more acoustic communication devices of the vehicle to generate an acoustic output if it has been determined that there is a need for communication between road user and vehicle, particularly if it has been determined that the road user is in the process of ascertaining whether be has been detected by the vehicle. The acoustic output can include human voice; abstract, nonhuman voice; and/or a transient sound event. In particular, the acoustic output can include a combination of transient and nontransient sound events. In this case, the nontransient sound events may be abstract and/or nonabstract. Furthermore, the transient sound events can include hum voice and/or a combination of abstract and/or nonabstract sound elements that act as nonhuman voice.
The acoustic output may be aimed particularly at indicating to the road user that be has been detected by the vehicle and/or in what function (concerning the traffic situation) the road user has been detected by the vehicle. Furthermore, the acoustic output may be aimed at indicating to the road user what intention the vehicle has in relation to a traffic situation (in particular a possible hazard situation or a situation in which the flow of traffic could be impaired).
The control unit described ensures that road users, in particular human road users, are identified by the vehicle, even without intervention by a driver of the vehicle, and are informed by the vehicle about the fact that they have been identified. By implication, this also means that the road user is provided with clarity in this regard in the case in which he has not been identified by the vehicle. The certainty about noticing and being noticed in road traffic typically results in an increase in safety in road traffic. Hence, the control unit can help to increase the safety of (possibly autonomous vehicles in road traffic, particularly in relation to human road users.
As already set out above, the control unit may be set up to ascertain a position of the road user (or positions of a multiplicity of road users). In addition, the control unit may be set up to generate a locally pinpointed acoustic output in the direction of the position of the road user. For this purpose, the one or more acoustic communication devices can include an acoustic group emitter having a multiplicity of individual emitters. The control unit may be set up to prompt the acoustic group emitter to generate the locally pinpointed acoustic output. The transmission of locally pinpointed acoustic outputs for one or more different road users can increase the unambiguity for the communication between the vehicle and the one or more road users. This in turn results in increased safety for autonomously driving vehicles.
The control unit may be set up to ascertain one or more properties in relation to the need for communication between road user and vehicle. The one or more properties in relation to the need for communication can indicate e.g. the urgency of a communication between road user and vehicle (between low and high urgency). Alternatively, or additionally, one or more properties in relation to the need for communication can indicate the content and/or the intention of the communication between road user and vehicle (e.g. a warning about a hazard situation, a piece of information about the intention/planned driving strategy of the vehicle, a request to the road user, etc.).
The control unit may be set up to generate or adjust the acoustic output on the basis of the one or more properties in relation to the need for communication. In particular, a parameter of an output sound can be adjusted. By way of example, an output sound can be adjusted in order to underline the urgency of the communication between vehicle and road user. By adjusting the acoustic output, it is possible to increase the quality of the communication between vehicle and road user further, resulting in increased safety for the autonomously driving vehicle.
The control unit may be set up take the need for communication as a basis for selecting a sound for the acoustic output from a multiplicity of predefined sounds, for generating a sound for the acoustic output by means of a sound algorithm from a multiplicity of predefined sound algorithms and/or for modulating a sound for the acoustic output. The provision of a multiplicity of predefined sounds and/or sound algorithms and also the provision of modulation options can allow precise communication by use of acoustic signals, which in turn increases safety for the autonomous vehicle.
The control unit may be set up to ascertain ambient noise in the environment of the vehicle. In particular, a frequency spectrum and/or a volume level for the ambient noise can be ascertained. The acoustic output can then be generated on the basis of the ambient noise. In particular, the acoustic output can be generated such that the acoustic output can still be noticed by the road user despite the ambient noise. This ensures reliable communication.
As set out above, the control unit may be set up to take the environment data as a basis for determining that there is a need for communication between the vehicle and a multiplicity of road users. Furthermore, the control unit may be set up to prompt the one or more acoustic communication devices of the vehicle to generate a multiplicity of acoustic outputs for communication with the corresponding multiplicity of road users. In particular, multiplicity of locally pinpointed, directional, acoustic outputs can be generated for the road users. It is thus possible to achieve univocal communication with multiple road users and further increased road safety for autonomously driving vehicles.
The control unit may be set up to take the environment data and possibly to take the position data as a basis for ascertaining one or more indicators for an intention of the road user. The one or more indicators for an intention of the road user may include e.g. a direction of movement of the road user relative to the vehicle (from which e.g. it is possible to infer that the road user wishes to cross the road ahead of the vehicle), a line of vision of the road user relative to the vehicle (looking right/left indicates e.g. that the road user wishes to cross a road), a road sign and/or a traffic installation in the environment of the vehicle and the road user (a zebra crossing indicates e.g. that a road user wishes to cross the road).
On the basis of the one or more indicators, it is then possible to determine that there is a need for communication between road user and vehicle. In particular, it is possible to determine that there is a hazard situation between vehicle and road user. In this case, a hazard situation may be e.g. a possible risk of collision between vehicle and road user. The control unit may then be set up to prompt the one or more acoustic communication devices of the vehicle to generate a (possibly further) acoustic output, e.g. in order to indicate to the road user how the vehicle will behave in relation to the hazard situation and/or in order to provide an instruction or a request for the road user so that the road user can adjust his behavior and in this way a safe encounter with the vehicle is rendered possible. Hence, particularly a human road user can obtain certainty about whether or not the vehicle presents a hazard. It is therefore possible to ensure increased safety, fluid traffic and increased acceptance in road traffic even for autonomous vehicles.
The control unit may be set up to ascertain what content needs to be communicated from the vehicle to the road user and/or how the content needs to be communicated from the vehicle to the road user. It is therefore possible to ascertain a communication strategy. This can be ascertained on the basis of the environment data and possibly on the basis of the vehicle data cited further below. The acoustic output can then also be generated on the basis of the content to be communicated and/or on the basis of how the content is to be communicated.
The control unit may be set up to ascertain vehicle data in relation to a driving strategy of the vehicle, in relation to a driving destination of the vehicle, in relation to a driving mode of the vehicle, in relation to an action by the occupant of the vehicle and/or in relation to a preference of an occupant of the vehicle. It is then also possible to take the vehicle data as a basis for determining whether there is a need for communication between road user and vehicle, and if need be what content needs to be communicated to the road user in what way.
According to a further aspect, a method for communication between a vehicle and a road user is described. The method includes the ascertainment of environment data on the basis of one or more environment sensors of the vehicle. Furthermore, the method includes the detection, on the basis of the environment data, of at least one road user in an environment of the vehicle. In addition, the method includes the determination, on the basis of the environment data and the driving strategy, of whether there is a need for communication between road user and vehicle. In particular, it is possible to take the environment data as a basis for determining that the road user is in the process of ascertaining whether be has been detected by the vehicle. The method additionally includes, when it has been determined that there is a need for communication between road user and vehicle, the selection of communication content and the generation of a communication strategy and, on the basis thereof, the generation of an acoustic output by one or more acoustic communication devices of the vehicle in order to communicate with the road user, e.g. in order to indicate to the road user that be has been detected by the vehicle, and/or in what function (e.g. as a pedestrian, as a cyclist, as another vehicle) the road user has been detected by the vehicle, and/or what intention the vehicle has in relation to the traffic situation (particularly a possible hazard situation or a situation in which the fluid flow of traffic is impaired), and/or what knowledge the vehicle has about the intention of the road user in relation to the traffic situation. The knowledge of the vehicle about the intention of the road user in relation to the traffic situation can be obtained e.g. on the basis of the one or more indicators for the intention of the road user.
According to a further aspect, a vehicle (particularly a road vehicle, e.g. an automobile, a truck or a motorcycle) is described that includes the control unit.
According to a further aspect, a software (SW) program is described. The SW program can be set up to be executed on a processor (e.g. on a control unit) and in order to carry out the method described herein as a result.
According to a further aspect, a storage medium is described. The storage medium can store an SW program that is set up to be executed on a processor and to carry out the method described herein as a result.
It should be noted that the methods, apparatuses and systems described in this document can be used either on their own or in combination with other methods, apparatuses and systems described in this document. Furthermore, the aspects of the methods, apparatuses and systems described in this document can be combined with one another in many and diverse ways.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of one or more preferred embodiments when considered in conjunction with the accompanying drawings.